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BRIEF  OUTLINE  OF  RESEARCH  FINDINGS 

ARO  Contract:  DAAG29-84-K-0156 
January  1,  1985  -  June  30,  1985 


A  considerable  effort  has  been  made  In  this  period  Investigating 
the  "new"  algorithm  for  linear  programming  of  Karmarkar.  An  extensive 
theoretical  and  numerical  development  of  the  algorithm  has  been  made.  The 
results  of  this  Investigation  will  be  published  In  the  near  future.  In 
summary  we  have  shown  that  Kamarkar's  algorithm  Is  a  synthesis  of  a 
projected  Newton  method  and  a  log  barrier  transformation.  A  separate  study 
of  Karmarkar's  method  by  a  graduate  student  has  been  published  [SOL  85-5). 
In  this  report  It  Is  shown  how  certain  practical  difficulties  with 
Karmarkar's  original  algorithm  can  be  overcome  without  any  loss  of  the 
theoretical  properties  of  the  algorithm* 

The  most  efficient  algorithms  for  nonlinear  constrained  optimization 
problems  use  a  sequential  quadratic  programming  approach.  In  certain  of 
these  algorithms  a  projection  of  the  Hessian  approximation  Is  recurred. 

The  definition  of  this  projection  Is  via  a  matrix  spanning  the  null-space 
of  the  Jacobian  matrix.  It  Is  vital  for  such  algorithms  that  this  matrix 
(whose  elements  are  functions)  have  certain  continuity  properties.  In 
report  SOL  85-1  we  show  how  matrices  spanning  the  null-space  may  be 
generated  efficiently. 

Optimization  almost  always  Involves  an  Interactive  process  between 
defining  the  problem  (model)  and  Its  solution.  In  report  SOL  85-2  this 
symbiotic  relationship  Is  discussed. 


BRIEF  OUTLINE  Of  RESEARCH  FINDINGS 


ARO  Contract;  DAAG29-84-K-0156 
July  1,  1985  -  Decenber  31,  1985 


In  the  thesis  work  of  Stephen  Hoyle  a  single  phase  active  set  method 
has  been  developed.  It  differs  from  other  active-set  methods  In  that  the 
current  iterate  may  violate  constraints  In  the  working  set.  One  conse¬ 
quence  of  allowing  violations  Is  that  the  Kuhn-Tucker  equations  may  become 
linearly  dependent.  A  key  finding  of  the  research  was  how  to  circumvent 
such  singularities.  It  Is  of  particular  Importance  that  the  techniques 
developed  are  applicable  to  large-scale  problems. 

Our  work  on  Interior  point  methods  for  linear  programming  has  con¬ 
tinued.  The  report  on  this  work,  SOL  85-11,  has  been  In  great  demand.  A 
prototype  algorltln  has  been  developed  and  tested  on  a  selection  of  medium 
to  large-scale  problems.  The  Initial  results  are  extremely  encouraging. 
Current  research  efforts  are  being  directed  at  Improving  the  efficiency  of 
solving  the  linear  least  squares  problem  that  occurs  at  each  Iteration  of 
the  method. 

Among  the  topics  discussed  In  the  paper  "Model  Building  and  Practical 
Aspects  of  Nonlinear  Programming”,  Is  how  the  QP  subproblems  posed  at  each 
Iteration  of  an  SQP  method  can  be  solved  with  high  efficiency  by  utilizing 
Information  available  from  the  previous  Iteration. 
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BRIEF  OUTLINE  OF  RESEARCH  FINDINGS 

ARO  Contract:  DAAG29-84-K-0156 
1  January  1986  ~  30  June  1986 

During  the  current  period  a  larger  number  of  reports  were  published 
than  usual.  Much  of  the  work  contained  In  the  reports  had  been  In  progress 
over  several  years.  For  example,  the  report  SOL  86-9  Is  the  Ph.D.  thesis 
of  Stephen  Hoyle.  This  report  may  be  viewed  as  a  first  step  In  developing 
a  general  purpose  QP  algorithm  for  large-scale  problems. 

The  report  86-8  describes  a  set  of  algorithms  for  computing  and 
modifying  the  LU  factors  of  a  sparse  rectangular  matrix.  Such  algorithms 
form  the  basic  building  blocks,  not  only  of  algorithms  for  large-scale 
optimization,  but  also  for  many  other  areas  of  scientific  computing. 

The  reports  86-1  and  86-2  are  users  guides  for  the  fortran  packages 
LSSOL  and  NPSOL  respectively.  Unlike  the  package  QPSOL  (SOL  84-6),  the 
routine  LSSOL  takes*  specific  advantage  of  the  problem  being  convex. 
Moreover,  the  problem  may  be  stated  In  the  form  of  a  least  squares. 
Consequently,  LSSOL  Is  able  to  solve  considerably  worst  conditioned 
problems  than  QPSOL.  Ill  conditioning  Is  a  common  feature  of  least  square 
problems.  The  new  version  of  NPSOL  Incorporates  LSSOL  In  addition  to  a 
number  of  ocher  Important  changes.  Perhaps  the  most  significant  change  Is 
that  gradients  are  no  longer  an  essential  requirement.  Work  Is  continuing 
on  NPSOL.  Currently  alternative  quasi-Newton  updates  are  being 
Investigated. 

In  report  86-6  the  theoretical  foundation  of  Che  meric  function  used 


in  NPSOL  is  given. 
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BRIEF  OOTLINE  OF  RESEARCH  FINDINGS 

ARO  Contract:  DAAG29-84-K-0156 
1  July  1986  -  31  December  1986 


The  research  on  numerical  optimization  has  been  in  two  principal  areas. 
First,  work  has  continued  on  the  development  and  Implementation  of  reliable 
algorithms  for  the  solution  of  constrained  least-squares  and  quadratic 
programming  problems. 

One  of  the  most  serious  questions  In  the  construction  of  methods  for 
constrained  least-squares  estimation  Is  the  treatment  of  singularity  in  the 
data  matrix.  Practical  problems  arising  in  statlsclcal  estimation, 
robotics  and  control  Invariably  have  singular  or  Ill-conditioned  data 
matrices.  These  matrices  may  cause  serious  problems  for  numerical 
algorithms.  Algorithms  must  be  able  to  solve  singular  problems  reliably 
and  efficiently. 

One  of  the  most  wldely-used  methods  for  singular  least-squares 
problems  Is  due  to  Stoer  (On  the  numerical  solution  of  constrained 
least-squares  problems",  SIAM  J.  on  Numerical  Analysis  8,  382-411,  1971). 
In  Stoer 's  method,  the  original  data  matrix  Is  overwritten  by  the  so-called 
complete  orthogonal  factorization  of  a  reduced  matrix  related  to  the 
projected  Hessian.  The  orthogonal  factorization  Is  then  updated  as 
constraints  enter  or  leave  the  working  set.  At  each  Iteration,  the 
updating  procedure  requires  some  procedure  for  estimating  the  rank  of  the 
reduced  matrix.  This  decision  Implicitly  defines  some  part  of  the 
factorization  as  being  "negligible"  and  small  elements  are  discarded. 
Unfortunately,  a  guaranteed  strategy  for  determining  the  numerical  "rank" 
of  a  matrix  Is  not  possible,  since  the  crucial  decision  about  when  a 
quantity  ls"negllglble"  depends  on  the  problem.  If  a  wrong  decision  is 
made  about  the  rank  and  Information  Is  erroneously  discarded,  the  computed 
solution  may  be  that  of  a  problem  that  differs  considerably  from  the 
original. 

A  new  method  for  constrained  least-squares  and  positive  semi-def inlte 
quadratic  programming  has  been  developed  that  Is  less  susceptible  to  the 
problems  of  rank  determination.  In  the  new  method,  a  decision  about  rank 
does  not  Imply  that  small  elements  in  a  reduced  matrix  are  assumed  to  be 
zero.  As  a  consequence,  the  problem  does  not  Implicitly  change  as  the 
Iterations  proceed.  Work  Is  In  progress  on  an  SOL  report  describing  the 
theoretical  details  of  the  algorithm.  The  new  method  has  been  Implemented 
In  an  updated  version  of  the  SOL  Fortran  package  for  constrained 
least-squares  and  quadratic  programming. 

The  second  area  of  active  research  concerns  the  development  of 
techniques  for  generating  test  problems  for  linear  programming,  quadratic 
programming  and  constrained  least-squares. 
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A  key  tool  In  the  development  of  algorithms  is  the  availability  of  a 
pool  of  realistic  non-trivial  test  problems.  The  theoretical  analysis  of 
algorithms  yields  much  information  concerning  efficiency  and  numerical 
stability.  However,  the  ultimate  test  is  how  the  algorithm  performs  in 
practical  situations.  An  algorithm  should  be  tested  not  only  in  its  final 
form,  but  also  throughout  its  development  stages.  In  many  cases,  the  only 
way  to  choose  between  seemingly  equivalent  strategies  is  to  select  that 
which  is  the  most  effective. 

During  the  last  contract  period,  a  problem  generator  has  been 
developed  for  positive-definite  and  indefinite  quadratic  programming. 

The  generator  has  a  number  of  unique  features.  First,  it  requires 
only  minimal  input  data,  the  dimension  of  the  problem,  the  number  of 
constraints  active  at  the  solution,  and  the  spread  of  singular  values  of 
the  constraints  and  the  Hessian  matrix  of  second  derivatives.  Second,  it 
is  possible  to  specify  the  range  of  singular  values  of  certain  key  metric 
projections  that  occur  in  the  bounds  on  the  absolute  perturbation  of  the 
solution  and  multipliers  induced  by  perturbations  in  the  data. 

We  anticipate  that  the  availability  of  the  problem  generator  will  have 
a  major  impact  on  research  in  constrained  least-squares  and  quadratic 
programming  at  SOL  and  other  centers  of  research. 
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